feat: support candidate and accepted drift stores - #234
Conversation
🤖 Codex PR Review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 04b8d201ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
8ca278b to
c29b037
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2033fe757d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5ebf192 to
d76f061
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d76f0611c6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 42f26c06dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 24a9fd2cdf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
2925ffb to
2b626fd
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2b626fd1f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
ba5badf to
c5556bf
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5556bf46e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Co-Authored-By: Codex <noreply@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a306ab9466
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return PerformanceStore( | ||
| cloud_bucket=bucket, | ||
| cloud_prefix=prefix, | ||
| local_root=Path(local_root) if local_root else None, |
There was a problem hiding this comment.
When --baseline-bucket is used without --baseline-local-root, this passes local_root=None, but PerformanceStore interprets None as DEFAULT_LOCAL_ROOT and load_latest_backtest() merges cloud keys with local keys. In cloud candidate runs the active store also writes candidate backtests to that default local root, so the accepted-baseline store can select a local candidate backtest instead of the accepted bucket baseline, defeating the candidate-vs-accepted comparison. Use an isolated or disabled local root for bucket-only baselines.
Useful? React with 👍 / 👎.
| dimensions=dimensions, | ||
| previous_status=DriftStatus(str(data["previous_status"])) if data.get("previous_status") else None, | ||
| baseline_param_set_id=str(data["baseline_param_set_id"]) if data.get("baseline_param_set_id") else None, | ||
| baseline_available=bool(data.get("baseline_available", True)), |
There was a problem hiding this comment.
When the new missing-baseline paths persist a placeholder result, DriftResult.to_dict() writes alert_suppressed=True, but this deserializer only restores the new baseline fields and leaves alert_suppressed at its default False. After PerformanceStore.load_latest_drift() round-trips one of these REVIEW placeholders it looks alertable again to any caller that consumes persisted drift results rather than the immediate run_drift_detection() return value, so baseline outages can leak back into alert/issue flows. Restore the persisted suppression flag when reconstructing the result.
Useful? React with 👍 / 👎.
| drifts = run_drift_detection(domain, store=store) | ||
| alerts = [d for d in drifts if d.status != DriftStatus.HEALTHY] | ||
| alerts = [d for d in drifts if d.status != DriftStatus.HEALTHY and not d.alert_suppressed] |
There was a problem hiding this comment.
When operators run quant-lifecycle autopilot or run_auto_pilot_cycle in a candidate/accepted-store rollout, this phase still invokes drift detection with only the candidate store, and I checked that neither the auto-pilot CLI nor run_auto_pilot_cycle accepts a baseline store or lineage policy. That means auto-pilot can optimize and create actions against candidate backtests while standalone drift/lifecycle compare to accepted baselines; thread the accepted baseline options through the auto-pilot drift and issue phases as well.
Useful? React with 👍 / 👎.
1 participant
Adds an optional baseline store to drift detection while preserving the default single-store behavior.